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Abstract. This study explores students’ perceptions about using Automated Writing 
Evaluation (AWE), Grammarly (a paid version), as a complementary instructional 
tool to teach and support writing from sources. Participants were second-year 
students (n=37) at a public university in Japan. After in-class tasks aimed at teaching 
paraphrasing, summarizing, and synthesizing, students completed a survey that 
measured their perceptions. Students had positive attitudes about Grammarly in 
general but had somewhat polarized opinions on how useful the tool is in teaching 
writing from sources and helping with plagiarism. 
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1. Introduction 


The integration of secondary sources challenges many inexperienced English as a 
Foreign Language (EFL) academic writers (Hirvela & Du, 2013; Liu, Lo, & Wang, 
2013). Undergraduate L2 students often struggle with paraphrasing, summarizing, 
synthesizing, as well as with appropriate attribution and referencing, which may 
reduce motivation in the L2 writing classroom. Research suggests it may also lead 
to unintentional plagiarism (Howard, 1995; Pecorari & Shaw, 2018). In the Asian 
context, students ‘receive limited exposure’ to textual borrowing strategies (in 
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Keck, 2014). Yet, these are essential when it comes to writing academic texts. To 
address these issues, in this exploratory study, the authors focused on the below. 


What are students’ perceptions about AWE tools and their use as a 
complementary instructional tool to teach and support writing from 
sources in an academic writing class? 


Grammarly was effective in reducing surface-level errors (Ghufron & Rosyida, 
2018), and students had positive perceptions about the tool (O’ Neill & Russell, 
2019). However, Grammarly’s in-built potential as a plagiarism checker® or text- 
matching software was used less in studies about writing from sources. 


2. Method 


The study was conducted during a 16-week EFL academic writing course at a 
Japanese university in 2019. Second-year students (n=37) were placed in three 
groups (majors, TOEFL ITP® scores) for weekly 90-minute classes. The overall 
goal of the course is to teach writing a 2000-word academic essay. A pre-survey 
measured students’ interests in English and asked about previous AWE experiences 
(Table 1). 


Table 1. Participants’ demographics 


Measure Item Frequency | Percentage 
English language 5-10 years 28 76% 
experience >10 years 9 24% 
Have you used any Yes* 13 35% 
other AWE tool? No 24 65% 
What is plagiarism? No answer 13 35% 
I don’t know 9 24% 
Answered 15 41% 


Note: *previously used AWE tools: Educational Testing Service (ETS) Criterion®, MS Spell Checker, and Grammarly; 
11 students used Grammarly. 


Three in-class tasks introduced and practiced paraphrasing (Week 4), summarizing 
(Week 5), and synthesizing (Week 6). Each writing task was presented to students 
in the first 30 minutes of the class. Students were then assigned homework, based 
on in-class tasks, to be submitted online within one week. 


5. https://www.grammarly.com/faq#toc0 
6. Test of English as a Foreign Language Institutional Testing Program 


184 


Student preferences: using Grammarly to help EFL writers... 


After the final homework, students completed a 48-item mixed-method survey (six 
point Likert type and five open-ended questions) to investigate their perceptions 
of using Grammarly to develop their ability to integrate sources in their writing. 
Due to the relatively small number of participants, Likert items were not combined 
into scales; thus, the response data were treated as ordinal data, and descriptive 
statistics were used (Lavrakas, 2008). For detecting trends in open-ended questions, 
KH Coder (Higuchi, 2016) was used. 


3. Results and discussion 


A snapshot of students’ overall perceptions is summarized in Table 2 (n=37). The 
reported percentages, median, and InterQuartile Range (IQR) show students agree 
on all questions, except about confidence. Most students (49%) do not feel more 
confident in their writing after doing in-class and homework tasks. It is possible 
that they became more aware of the gaps in their knowledge both in terms of 
linguistic and rhetorical aspects of writing after using Grammarly. 


Table 2. Students’ perceptions about Grammarly 


vo 
a > o 
cP) cP) ia 
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al Se | |€2 (2. le |S. [do 
i = op on Eo |B o ao = 
ve Sa g a 2 2 so |x 
x re a of 8 a Cie] 
ia 2B =) = ay = on op + 60 = 
—q= ans | NS |Na |< na S 
I know how to revise 0% 0% 3% 30% (54% |14% | 5(1) 


my essay based on 
Grammarly’s feedback. 
Grammarly is user-friendly. (0% 0% 0% 16% (62% (22%  5(0) 
I think ‘correctness’ 0% 0% 0% 16% 43% (41% | 5(1) 
alerts are useful. 
Grammarly developed my 0% 3% 16% 51% (24% |5% 4(1) 
language long-term as I could 
understand grammar more. 
Rewriting, after 0% 5% 24% 51% (16% |3% 4(1) 
receiving Grammarly 
feedback, increased my 
motivation to write. 

I became more confident 0% 11% (49% (30% 11% 0% 3(1) 
in writing because 
of how Grammarly 
was used in class. 
Grammarly made me feel 0% 5% 19% 43% (24% |8% 4(1) 
more confident about handing 
the 2000-word essay in. 
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These answers were corroborated by analyzing open-ended questions. Figure | 
presents the results of the co-occurrence of words: patterns are detected based 
on the degree of modularity (Higuchi, 2016). Patterns are detected via modules, 
grouped circles of the same color: Grammarly is easy to use and understand and 
can help find mistakes and notice instances of plagiarism. 


Figure 1. Grammarly advantages, the co-occurrence of words, Q34. n=37, N31 


D65 D.14 
: AO —$§$ fer 
notite 2 
| Og. 
7 ry feedback 
gar i? 
/ ~~. ~ get Subgraph: 
| ena avoid j Ho. mo 
| 7 mae \ /| 02 fos 
eo | L | B03 
—~ : _seafe Frequency: 
grammar call & das | 


point }é > tn 


_—— dolor 


Cis 
C) x 
( )« 


want 


ol Be 


Fuse) 
X 


seal a 


Table 3 (n=37, *n=36) shows perceptions about Grammarly as an instructional 
tool to help with writing from sources and avoiding plagiarism. Overall, students 
agree that Grammarly is useful for avoiding plagiarism, and it helped them with 
enhancing all three textual borrowing skills, although some of their answers are 
more scattered across the range of six possible answers. 


On average, most students, 24 (65%), used Grammarly once a week. When asked 
about when/where and how they used this tool, students showed the highest levels 
of disagreement (Table 4). 
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Table 3. Students’ perceptions: Grammarly as an instructional tool to help with 
writing from sources 


vo 
ao Se o ia 
a of of & is s 
ae So la las [22 |2 |82 |se 
I can understand alerts related | 3% 11% (16% |24% |38% |8% 4(2) 
to ‘plagiarism’ alerts. 
When Grammarly flags 3% 22% 16% (35% |22% |3% 4(1) 
plagiarism, I know how to 
revise the part written from 
sources based on feedback. 
Grammarly can enhance 0% 0% 19% (51% |24% |5% 4(1) 
paraphrasing skills. 
Grammarly can enhance 0% 3% 35% (41% 119% |3% 4(1) 
summarizing skills. 
Grammarly can enhance 0% 8% 36% (47% |8% 0% 4(1) 
synthesizing skills.* 


Table 4. Frequency and way of use 


vo 
i= ao gv o > 
15 e EB 5, = E = am || Sp ? rs) a Z 
2¢ es if (Be |BE |B [28 [36 
= as |A As |nR |< aa |zs 
I used Grammarly 30% |16% |11% (11% (19% 14% | 3(4) 
outside AW class. 
I used Grammarly only 16% |16% (22% |22% (14% 11% | 3(2) 
when my instructor 
asked me to do so. 
When using Grammarly, 5% 24% 19% |32% |19% (0% 4(2) 
I read extended 
explanations of errors. 


Analysis of an open-ended question about the use of Grammarly as a tool for 
writing from sources and plagiarism (Figure 2) shows several patterns: easy 
to use; plagiarism is difficult to avoid, but Grammarly can help; students can 
notice things that they did not notice before. Students did not read Grammarly’s 
explanations, and only paraphrasing was mentioned. Some of the types of 
the student responses (Figure 2): “I had used the same sentences without 
noticing”;“It’s difficult to avoid plagiarism perfectly but feedback helped me to 


understand”; and “It made me think I have to paraphrase”. 
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Figure 2. Grammarly and writing from sources, the co-occurrence of words, Q44. 
n=37, N21 D60 D.286 
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4. Conclusions 


This exploratory study investigated students’ perceptions regarding Grammarly as 
a complementary instructional tool to teach and support writing from sources. It 
was the first step in the potential implementation of the use of AWE within an 
EFL academic writing course. As in previous research, participants had positive 
perceptions of Grammarly and found it useful in addressing shortcomings in their 
grammar knowledge, word usage, style, and mechanics of writing. Students found 
Grammarly to be a beneficial instructional tool that can help avoid plagiarism and 
writing from sources. Next is to look at how this kind of Grammarly use affected 
student’s revisions by looking at the writing samples collected in the study. There 
are several implications for the practitioners: when deciding to use the tool, 
instructors should think about student’s skill levels — if they have enough meta- 
linguistic knowledge to understand the software output; focus on paraphrasing 
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only; spend more in-class time demonstrating Grammarly’s use; and use it in 
combination with the instructor’s feedback. 
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